REMARKS 

This application has been carefully reviewed in light of the final Office 
Action dated March 11, 2008. Claims 1, 3 to 5, 11 to 15, 18 to 21, 23, 26 to 28, 34 to 38, 
41 to 54 and 56 are in the application, of which Claims 1, 23 and 56 are in independent 
form. Reconsideration and further examination are respectfully requested. 

The Specification was objected to for allegedly failing to provide proper 
antecedent basis for the claimed subject matter. Specifically, the Office Action alleged that 
the phrase "computer readable medium" is not found to have proper antecedent basis in the 
specification, and thus, could be interpreted to include non-statutory subject matter, such as 
"transmission channels". In response, the claims have been amended to replace the phrase 
"computer readable medium" with "computer readable storage medium", so as to 
distinguish the claimed "medium" from "transmission channels". Accordingly, 
reconsideration and withdrawal of the objection are respectfully requested. 

Claims 23, 26 to 28, 34 to 38, 41 to 44 and 50 to 54 were rejected under 35 
U.S.C. § 101 for allegedly being directed to non-statutory subject matter. Without 
conceding the correctness of the rejection, Claim 23 has been amended to recite a 
"computer readable storage medium", in keeping with the guidelines at MPEP § 2106.01. 
Accordingly, reconsideration and withdrawal of the rejection are respectfully requested. 

Claims 1, 3 to 5, 11, 13, 15, 18, 20, 21, 23, 26 to 28, 34, 36, 38, 41, 43 to 54 
and 56 were rejected under 35 U.S.C. § 103(a) over U.S. Patent No. 6,687,878 (Eintracht) 
in view of U.S. Patent No. 5,821,931 (Berquist) in view of U.S. Patent Application 
Publication No. 2002/0054059 (Schneiderman) in view of U.S. Patent No. 6,597,800 
(Murray) and further in view of U.S. Patent No. 6,41 1,313 (Conlon). Claims 12 and 35 
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were rejected under 35 U.S.C. § 103(a) over Eintracht in view of Berquist in view of 
Schneiderman in view of Murray in view of Conlon and further in view of U.S. Patent No. 
6,021,221 (Takaha). Claims 14 and 37 were rejected under 35 U.S.C. § 103(a) over 
Eintracht in view of Berquist in view of Schneiderman in view of Murray in view of 
Conlon and further in view of U.S. Patent No. 6,616,701 (Doyle). Claims 19 and 42 were 
rejected under 35 U.S.C. § 103(a) over Eintracht in view of Berquist in view of 
Schneiderman in view of Murray in view of Conlon and further in view of U.S. Patent No. 
6,976,229 (Balabanovic). Reconsideration and withdrawal of the rejections are 
respectfully requested. 

Claims 1 and 23 

The present invention, as defined by Claims 1 and 23, involves annotating 
an image by both automatically and manually placing bounded regions on the image, and 
by dragging icons associated with metadata labels onto the bounded regions. More 
specifically, a plurality of metadata labels is extracted from an existing database of 
metadata labels to form a list of metadata labels. Each of the metadata labels in the list is 
associated with at least one of a plurality of icons, and the plurality of icons are displayed. 
Each of the icons is labeled with one or more of the metadata labels with which the icon 
was associated. The image is displayed adjacent to the displayed plurality of labeled icons, 
wherein the metadata labels are generated prior to having knowledge of the content of the 
image. One or more subjects are detected within the image, using an image detection 
method, to form one or more automatically placed bounded regions within the image. 
Each of the automatically placed bounded regions substantially surrounds at least one of 
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the detected subjects within the image. In addition, one or more manually placed bounded 
regions are formed within the image in response to user input. Selection of at least one of 
the displayed plurality of labeled icons is detected, and the selected icon is dragged to the 
image, such that at least one of the bounded regions is changed upon the selected icon 
being dragged over the bounded region, in order to emphasize the bounded region. The 
selected icon is dropped within the changed bounded region, which corresponds to a 
selected subject within the image. The one or more metadata labels associated with the 
selected icon are linked with a description of the location of the selected subject within the 
image, and the linked one or more metadata labels and the description are stored as an 
annotation of the image. 

Applicants submit that the applied references, alone or in any permissible 
combination, are not seen to disclose or to suggest the features of Claims 1 and 23. In 
particular, the applied references are not seen to disclose or to suggest at least the features 
of forming one or more automatically placed bounded regions and one or more manually 
placed bounded regions, wherein to form the automatically placed bounded regions there is 
a detection of one or more subjects within an image, using an image detection method, 
each automatically placed bounded region substantially surrounding at least one of the 
detected subjects within the image, and wherein to form the manually placed bounded 
regions, there is a response to user input. 

Murray is seen to disclose a process for automatic recognition of a target 
object from live infrared or visible light image data in real time. Murray's automatic target 
recognition process is intended to recognize objects in a field of view, such as in a 
surveillance role to identify the presence of people or vehicles in a high security area. 
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Automatic target recognition processing may also be employed for pattern recognition, 
such as to identify handwriting, fingerprints, or printed text. Murray's automatic target 
recognition process involves subjecting an image to primary segmentation in which the 
image is divided up into one or more primary homogeneous regions, each approximating to 
an object of interest, and enclosing the segmented pixels forming a primary homogenous 
region in a rectangular bounding box. 

While Murray is seen to disclose automatic recognition of a target object 
from live data in real time, Murray is believed to be silent on a user manually forming a 
bounded region within an image which corresponds to a selected subject within the image. 
In fact, because Murray is intended to provide improved accuracy of automatic target 
object recognition performed on live data in real time, Murray is believed to teach away 
from manually forming bounded regions in response to user input. Specifically, Murray's 
apparatus and process are intended to provided improved accuracy of target object 
recognition and identification for surveillance in a high security area, or for pattern 
recognition to identify handwriting, finger prints, or printed text. Since automatic 
recognition of target objects to identify the presence of people or vehicles in a high security 
area, and handwriting, finger print, or printed text pattern recognition are believed to be 
typically performed without a response to user input, Murray is believed to teach away 
from manually forming bounded regions in response to user input. 

Doyle is seen to disclose a user interactively associating program actions 
with objects in multi-dimensional image data, so that the program action is performed in 
response to a user clicking on an object in the image using a mouse. The multi- 
dimensional image can be a GIF or a JPEG file of an image, or an MPEG file of images. 
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Objects in the image data are interactively outlined by presenting the user with a two- 
dimensional representation of the multi-dimensional image, and a program action is 
associated for each object in the image data. The program action can display explanatory 
text related to the object in the image or provide a link to a particular Web page. For 
example, a multi-dimensional image, such as an image of an entire anatomical body, could 
be presented to a user, and the user can select a general area of the image using a mouse. 
In response to the user selection, a detailed view of the related region could be displayed to 
the user. The user would then select a particular location in the image and receive 
additional information related to the selected location. Thus, Doyle is seen to disclose 
interactively outlining objects in multi-dimensional image data so that program actions can 
be associated with the outlined objects. 

However, Doyle is believed to be silent on detecting subjects within the 
image using an image detection method, much less automatically placing bounded regions 
within the image that substantially surround detected subjects within the image. 

Applicants submit that it would not have been obvious to one of ordinary 
skill in the art at the time of Applicant' s invention to combine the interactive outlining of 
Doyle with the automatic target recognition of Murray, in part, because the disclosure of 
Murray is believed to teach away from manually forming bounded regions in response to 
user input, as described above. 

Moreover, it is noted that the current rejection of Claims 1 and 23 relies on 
a five-way combination of references, namely, Eintracht, Berquist, Schneiderman, Murray 
and Conlon. Applicants respectfully submit that the USPTO has not articulated an 
adequate rational to explain why those of ordinary skill would have been prompted to 
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identify particular elements in each reference, and then to combine these elements in the 

way that the Office Action does. See KSR International: 

"[A] patent composed of several elements is not proved 
obvious merely by demonstrating that each of its elements was, 
independently, known in the prior art. Although common sense directs one 
to look with care at a patent application that claims as innovation the 
combination of two known devices according to their established functions, 
it can be important to identify a reason that would have prompted a person 
of ordinary skill in the relevant field to combine the elements in the way the 
claimed new invention does, [inventions in most, if not all, instances rely 
upon building blocks long since uncovered, and claimed discoveries almost 
of necessity will be combinations of what, in some sense, is already 
known." KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385, 1390 
(2007). 



Here, the Office Action identified alleged "benefits" of various 
combinations without also identifying that those of ordinary skill would have recognized 
those benefits at the time of the invention, and would have some apparent reason for 
wanting the benefit of one reference in the context of another. It is therefore respectfully 
submitted that the rejection of Claims 1 and 23 should be withdrawn for failure to establish 
the legal predicate required for an obviousness rejection. 

Moreover, and as demonstrated above, it is believed that there are 
technological deficiencies in the rejection. Specifically, Murray and Doyle, alone or in any 
permissible combination of applied references, are not believed to disclose or to suggest at 
least forming one or more automatically placed bounded regions and one or more manually 
placed bounded regions, wherein to form the automatically placed bounded regions there is 
a detection of one or more subjects within an image, using an image detection method, 
each automatically placed bounded region substantially surrounding at least one of the 
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detected subjects within the image, and wherein to form the manually placed bounded 
regions, there is a response to user input. 

In view of the foregoing amendments and remarks, independent Claims 1 
and 23, as well as the claims dependent therefrom, are believed to be in condition for 
allowance. 

Claim 56 

The present invention, as defined by Claim 56, involves annotating an 
image by both automatically and manually placing bounded regions on the image, and by 
dragging a representation of a metadata label onto the bounded regions. More specifically, 
a plurality of metadata labels is extracted from an existing database of metadata labels to 
form a list of metadata labels, and a representation of each of the metadata labels in the list 
is displayed. The image is displayed adjacent to the displayed representations of metadata, 
wherein the metadata labels are generated prior to having knowledge of the content of the 
image. One or more subjects are detected within the image, using an image detection 
method, to form one or more automatically placed bounded regions within the image. 
Each of the automatically placed bounded regions substantially surrounds at least one of 
the detected subjects within the image. In addition, one or more manually placed bounded 
regions are formed within the image in response to user input. Selection of at least one of 
the displayed representations of metadata labels is detected, and the selected representation 
is dragged to the image, such that at least one of the bounded regions is changed upon the 
selected representation being dragged over the bounded region, in order to emphasize the 
bounded region. The selected representation is dropped within the changed bounded 
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region, which corresponds to a selected subject within the image. The one or more 
metadata labels associated with the selected representation are linked with a description of 
the location of the selected subject within the image, and the linked one or more metadata 
labels and the description are stored as an annotation of the image. 

For the reasons discussed above with respect to Claims 1 and 23, Applicants 
submit that the applied references, alone or in any permissible combination, are not seen to 
disclose or to suggest the features of Claim 56, and in particular, are not seen to disclose or 
to suggest at least the features of forming one or more automatically placed bounded 
regions and one or more manually placed bounded regions, wherein to form the 
automatically placed bounded regions there is a detection of one or more subjects within an 
image, using an image detection method, each automatically placed bounded region 
substantially surrounding at least one of the detected subjects within the image, and 
wherein to form the manually placed bounded regions, there is a response to user input. 

In view of the foregoing amendments and remarks, independent Claim 56, 
as well as the claims dependent therefrom, is believed to be in condition for allowance. 
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CONSIDERATION OF INFORMATION DISCLOSURE STATEMENT 



Page 2 of the Office Action indicates the Examiner' s continual refusal to 
consider properly- submitted information that he himself asked for. In the Information 
Disclosure Statement dated March 10, 2006, Applicants cited art from a European Search 
Report, and provided a copy of the European Search Report in compliance with the 
requirement for a concise explanation of relevance. In the Office Action dated July 19, 
2006, the Examiner stated that the references cited in the aforementioned Information 
Disclosure Statement had not been considered because the Form PTO-1449 did not list the 
European Search Report. In a Response dated October 19, 2006, Applicants stated that the 
European Search Report was submitted as a source of the references cited in the March 10, 
2006 Information Disclosure Statement, and that the European Search report did not have 
to be listed. In the Office Actions dated April 26, 2007 and October 29, 2007, the 
Examiner insisted on the submission of a Form PTO-1449 listing the European Search 
Report. Accordingly, Applicants submitted an Information Disclosure Statement on 
January 29, 2008 listing the European Search Report. The Examiner now refuses to 
consider this information still, even though it was requested by him specifically. 

The stated reason for refusal is that the Applicant "failed to [sic, list?] its 
pertinent pages." In its entirety, the European Search Report is two pages long, and it is 
respectfully requested that these two pages be considered. If there is continued difficulty in 
consideration of the information that the Examiner himself has requested, the Examiner is 
respectfully requested to contact the Applicants' undersigned attorney at (714) 540-8700, so 
as to determine exactly what is needed to obtain consideration of the Information 
Disclosure Statements dated March 10, 2006 and January 29, 2008. 
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No other matters being raised, it is believed that the entire application is 
fully in condition for allowance, and such action is courteously solicited. 

Applicants' undersigned attorney may be reached in our Costa Mesa, 
California office at (714) 540-8700. All correspondence should continue to be directed to 
our below-listed address. 

Respectfully submitted, 



/Shant Tchakerian #61.825/ 
Shant H. Tchakerian 
Attorney for Applicants 
Registration No.: 61,825 



FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 101 12-3800 
Facsimile: (212) 218-2200 
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